Visual Appearance of Matte

All visual sensors, biological and arti cial, are nite in resolution by necessity. This causes the e ective re ectance of surfaces in a scene to vary with magni cation. A reectance model for matte surfaces is described that captures the e ect of macroscopic surface undulations on image brightness. The model takes into account complex physical phenomena such as masking, shadowing and interre ections between points on the surface, and is shown to accurately predict the appearance of a wide range of natural surfaces. The implications of these results for human vision, machine vision, and computer graphics are demonstrated using both real and rendered images of three-dimensional objects. In particular, extreme surface roughness can cause objects to produce silhouette images devoid of shading, precluding visual perception of object shape. Painters and sculptors are known to exploit their instinct for the interaction between light and materials[1, 2] to convey compelling shape cues to the observer [3]. Re ection of light by materials may be viewed as the rst fundamental process in visual perception by human or machine. All re ectance mechanisms can be broadly classi ed into two categories: surface and body. In surface re ection, light rays are re ected at the interface between the surface medium and air. For very smooth surfaces, this results in specular or mirror-like appearance, the viewed surface producing a clear virtual image of its surroundings that is geometrically distorted when the surface is not planar [4]. As the surface gets rougher, the virtual image begins to blur, altering surface appearance from shiny to glossy, and even di use for high roughness values. Surface re ection is common, for instance, in metals. In body re ection, incident light rays penetrate the surface and are scattered around due to re ections and refractions caused by inhomogeneities within the surface medium. Some of this light energy may be absorbed by the surface or transmitted through it. The remaining nds its way back to the interface to re-emerge as body re ection. The random nature of sub-surface scattering causes emerging light rays to be distributed in a wide range of directions, giving the surface a matte appearance. Body re ection plays a dominant role in materials like clay, plaster, concrete, and paper. In many natural materials however, both surface and body mechanisms coexist and together determine nal visual appearance. Mathematical models for both re ection mechanisms, based on physical and geometrical optics, have been studied extensively. For body re ection, numerous models have been suggested for the scattering process [5, 6, 7]. Among these, Lambert's law [5], proposed in 1760, remains the most widely used in visual psychophysics [8], computational vision [9], remote sensing [10], and computer graphics [11]. It predicts that the brightness, or radiance, Lr, of an ideal matte surface point is  cos i, where , the albedo or re ectivity, represents the fraction of the total incident light re ected by the surface, and i is the incidence angle between the surface normal and the illumination direction. The popularity of Lambert's model can be attributed to its ability to predict with a fair degree of accuracy the appearance of a large spectrum of realworld materials. Another reason is undoubtedly its simple mathematical form, which lends 1 itself to numerous interesting appearance properties; for theoreticians and practitioners alike, the use of Lambert's law is a temptation di cult to resist. Both reasons have led to its widespread use in understanding and emulating perception of important visual cues such as shading. The most appealing aspect of Lambert's law is its prediction that the brightness of a scene point is independent of the observer's viewpoint. This in turn can be exploited to establish that a scene point illuminated by several light sources can be viewed as illuminated by a single source whose intensity and direction are given simply by the centroid of all the sources. Furthermore, the surface normal and albedo of a scene point can be uniquely determined from its brightness values measured using three known illuminants [12]. The simplicity of Lambert's law permits even the analysis of complex high-order phenomena such as interre ections [13, 14], the bouncing of light rays between mutually visible points on a concave surface. In the presence of interre ections, a surface continues to behave exactly like a Lambertian one without interre ections but with a di erent set of normals and albedo values [15]. Alas, our visual world limits the scope of Lambert's model. While it does well in describing sub-surface scattering in a large variety of materials, it fails to describe the ubiquitous interplay between surface undulations and image resolution (Figure 1). Visual processing by humans and machines must rely on nite-resolution sensors. Photoreceptors of the retina and pixels in a video camera are both by necessity nite-area detectors; light intensity can be recorded only by counting photons collected in buckets of measurable size. This nite resolution, along with the optical point spread [16] inherent to any imaging system, cause each receptor to receive light not from a single point but rather from a surface area in the scene, this area increasing as the square of the distance of the surface from the eye or the camera (Figure 1b). Often, substantial macroscopic ( wavelength of the incident light) surface roughness is projected onto a single detector, which in turn produces an aggregate brightness value. Whereas Lambert's law may hold well when observing a single planar facet (near sight), a collection of such facets with di erent orientations (far sight) is guaranteed to violate Lambert's law. The primary reason is the variation in foreshortened 2 facet areas under motion of the observer (Figure 2a). Analysis of this phenomenon has a long history and can be traced back almost a century. Past work has resulted in empirical models [17, 18] designed to t experimental data as well as theoretical results derived from rst principles [19, 20, 21]. Much of this work was motivated by the non-Lambertian re ectance of the moon [22, 23, 24]. Unfortunately, these models are severely limited in scope either by the speci c surface geometry assumed or by their inability to predict brightness for the entire hemisphere of source and sensor directions. A new re ectance model has been developed that describes the relation between macroscopic surface roughness and sensor resolution. The surface patch imaged by each sensor detector is modeled as a collection of numerous long symmetric V-shaped cavities (Figure 2b); each cavity has two planar Lambertian facets with opposing normals, facet normals are free to deviate from the mean surface normal, and all facets on the surface have the same albedo . It is assumed that the V-cavities are uniformly distributed in orientation a (azimuth angle) on the surface plane, whereas facet tilt a (polar angle) is normally distributed with zero mean and standard deviation , the latter serving as a roughness parameter1. This isotropic surface model has been previously used to study surface re ection from rough surfaces [25], and is invoked here to achieve mathematical tractability2. When = 0, all facet normals align with the mean surface normal, producing a planar patch that obeys Lambert's law. However, as increases, the V-cavities get deeper on the average, and the deviation from Lambert's model increases. 1Roughness, as de ned here, is a purely macroscopic property. It is not indicative of the microscopic structure of the surface with is assumed to cause the surface to be locally Lambertian. 2Natural surfaces clearly profess a wide spectrum of macroscopic surface geometries. For deriving re ectance, a speci c surface model must be assumed. In the past, several surface models [19, 20] have been found to be di cult to manipulate during re ectance model derivation. In the context of body re ection, radiance in the direction of the observer must be evaluated as an integral over all visible points on the surface patch projecting onto a single sensor detector. Since integration has the e ect of weighted averaging, it turns out that several surface models are capable of capturing the primary re ectance characteristics of the surface. The V-cavity model [21] was chosen as it facilitates the analysis of pertinent radiometric and geometric phenomena and hence results in a re ectance model that performs fairly well for a large variety of real surfaces. 3 The re ection model captures not only the foreshortening of individual facets (Figure 2a), but also masking, shadowing, and interre ections (up to two bounces) between adjacent facets [26]. The brightness of a surface patch is expressed as the integral of facet brightness over all facet normals. This integral is cumbersome to evaluate and must be broken into components representing facets that are masked, shadowed, masked and shadowed, and neither masked nor shadowed. The complexity of the integral is easily seen by imagining the di erent masking and shadowing conditions that arise as a single V-cavity is rotated in the surface plane. A solution to the integral was arrived at by rst deriving a basis function for each component of the integral, and then nding coe cients for the bases through extensive numerical simulations [26]. The accuracy of the model was veri ed by matching model predictions with re ectance measurements from natural surfaces, such as, plaster, sand, and clay (Figure 3). In all cases, predicted and measured data were found to be in strong agreement. A systematic increase in brightness is observed as the sensor moves towards the illuminant; this backscattering is in contrast to Lambertian behavior where brightness is constant and independent of sensor direction, and also in contrast to surface re ection where a peak in brightness is expected in the vicinity of the specular direction [25]. For applications where simplicity is desired over high precision, approximations were made to arrive at this qualitative model: L( r; i; r i; ; ) = E0 cos i(A+ BMax 0; cos ( r i) sin tan ) ; (1) A = 1:0 0:5 2 2 + 0:33 ; B = 0:45 2 2 + 0:09 ; where E0 is the intensity of the source, ( r; r) and ( i; i) are the observer and illuminant directions in a coordinate frame with its z-axis aligned with the surface normal, and = Max( r; i) and = Min( r; i). The developed model may be viewed as a generalization of Lambert's law, which is simply an extreme case with = 0. The model has direct implications for shape recovery in machine vision [26] and for realistic rendering in computer graphics [28]. Further, it provides 4 a rm basis for studying visual perception of three-dimensional objects. To illustrate this, several objects were constructed from materials such as porcelain and stoneware, and their digital images shown to closely match synthetic ones rendered using the model (Figure 4). Both real and rendered shadings are seen to vary synchronously, and signi cantly, with macroscopic roughness. These experiments have led to a curious observation: The model predicts that for very high macroscopic roughness, when the observer and the illuminant are close to one another, all surface normals will generate approximately the same brightness. This implies that, a three-dimensional object, irrespective of its shape, will produce nothing more than a silhouette with constant intensity within. In the case of polyhedra, edges between adjacent faces will no longer be discernible (Figure 4a), and smoothly curved objects will be devoid of shading (Figure 1a). This visual ambiguity may be viewed as a perceptual singularity in which interpretation of the three-dimensional shape of an object from its image is impossible for both humans and machines. This phenomenon o ers a plausible explanation for the at-disc appearance of the full moon (Figure 4e). 5 (a) Eye Camera Surface (b) Figure 1: (a) Digital images of two surface patches illuminated from the same direction. The strong shading of the right patch leads the observer to perceive a cylindrical surface with a vertical axis. In contrast, the left patch has fairly uniform brightness and the lack of shading seems to suggest a planar surface. The actual shapes of the surfaces are identical. Both patches are clipped from images (512x480 pixels) of cylindrical vases. On the left is a real clay vase with very rough exterior that gives it a at appearance. The right vase has identical shape but is rendered using Lambert's model [5] for body re ection. Lambert's law predicts strong shading and drives brightness at the occluding boundaries to zero. While it predicts the re ectance of several natural surfaces with adequate accuracy, it fails to capture the interplay between macroscopic surface roughness and sensor resolution. (b) Retina of the human eye [16] and solidstate sensors in video cameras have nite-size receptors that aggregate brightness from areas rather than points in the scene. The area projected onto a single receptor increases as square of surface distance from the sensor. In a typical CCD camera used with a 25 mm lens, each pixel images a foreshortened area of 9 mm2 at a distance of 5 m, or 144 mm2 at 20 m. Clearly, large amounts of macroscopic undulations can project onto a single pixel. 6 Bright Dark Source Observer Bright Dark Bright Dark Observer (a) d a d A â θa n̂ (b) Figure 2: (a) A single V-cavity used to illustrate why a collection of Lambertian facets with di erent orientations does not obey Lambert's law. When the cavity is illuminated from the right, the smaller incidence angle for the left facet makes it brighter than the right one. For an observer on the left, the foreshortening of the left facet is greater than of the right one and a larger fraction of the cavity is dark. As the observer moves right, towards the illuminant, the fraction of the brighter area increases, causing the aggregate brightness of the V-cavity to rise. (b) A re ectance function is derived by modeling a surface patch as a collection of V-cavities (da dA) with di erent facet normals (â). 7